Automatic Fairness Testing of Neural Classifiers Through Adversarial Sampling

Although deep learning has demonstrated astonishing performance in many applications, there are still concerns about its dependability. One desirable property of applications with societal impact is fairness (i.e., non-discrimination). Unfortunately, discrimination might be intrinsically embedded into the models due to training data. As a countermeasure, testing systemically identifies discriminatory samples, which can used retrain model and improve model’s fairness. Existing approaches however have two major limitations. First, they only work well on traditional machine poor (e.g., effectiveness efficiency) models. Second, simple structured tabular) data not applicable for domains such as text. In this work, we bridge gap by proposing scalable effective approach systematically searching samples while extending existing address more challenging domain, i.e., text classification. Compared state-of-the-art methods, our employs lightweight procedures like gradient computation clustering, significantly effective. Experimental results show that average, explores search space much effectively (9.62 2.38 times than methods respectively tabular datasets) generates (24.95 2.68 times) within same reasonable time. Moreover, retrained reduce 57.2 60.2 percent average.